A Natural User Interface to Control an Augmented Reality Application in the CAVE at the HTW Berlin

This paper introduces a Natural User Interface (NUI) that enables interaction in a Cave Automatic Virtual Environment (CAVE). The tracking of the head and hand in a CAVE usually takes place with the help of magnetic tracking systems or marker-based optical tracking systems. In both cases the user has to wear a number of instruments. In contrast, the CAVE at the HTW Berlin uses two depth detection cameras (currently Microsoft Kinect) and the data provided is then merged. This records the user's entire skeleton without the use of instruments and it allows applications to be controlled by a NUI. CAVE systems are usually used for virtual reality applications. The application described in this case visualises the CAVE itself, creating a spatial augmented reality system in which the user can interact directly with the real CAVE. The physical CAVE walls are aligned to the virtual walls so that the user can touch the walls and receive realistic haptic feedback.